Is Aristotle's Natural Philosophy objective?¶

Introduction¶

The scientific method is one of mankind's greatest creation. It allowed us to make observations about the world around us and discern what is fact and what is fiction, making way to great advancements in humanity's quality of life. Natural philosophy is well known as the predecessor to the scientific method. In fact, what we know as science today was simply called natural philosophy in the past. There didn't used to be a distinction.

Science is considered to be pretty objective in method. We have frameworks put in place like double-blind experimentation to reduce the number of errors due to human error or bias. Most people will also agree that a well written scientific article in a journal will be more objective than an article in a magazine. Can we use methods of machine learning in natural language processing to see if natural philosophy is similarly objective compared to other schools of thought?

The History of Philosophy dataset will be used to make the analysis, focusing specifically on Aristotle's body of work, since he has written texts on natural philosophy and also other topics such as logic and politics with which we can make a comparison on objectivity. The subset of works will be put into a topic modelling algorithm to try and discern which sentences are part of his works on natural philosophy, and then put through a sentiment analysis algorithm measuring the subjectivity of the sentences.

In [2]:
#Setup
#Operations
import pandas as pd
import numpy as np
import sys
import os

#Visualization
import seaborn as sns
import matplotlib.pyplot as plt
from wordcloud import WordCloud
from pprint import pprint
import pyLDAvis
import pyLDAvis.gensim_models as gensimvis

#NLP Cleaning
import spacy
from spacy.lang.en import English
import en_core_web_md
import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize

#NLP Algorithms
import gensim
from gensim.models import CoherenceModel
import gensim.corpora as corpora
from textblob import TextBlob

/opt/anaconda3/lib/python3.9/site-packages/scipy/sparse/sparsetools.py:21: DeprecationWarning: `scipy.sparse.sparsetools` is deprecated!
scipy.sparse.sparsetools is a private module for scipy.sparse, and should not be used.
  _deprecated()
In [3]:
#To allow for calling local functions
path_def='/Users/safiraraharjo/Documents/GitHub/ads-spring2023-project1-safiraharjo/'
sys.path.insert(0, path_def+'lib')

Topic Modelling¶

Topic Modelling is the task of using unsupervised learning to extract the main topics (represented as a set of words) that occur in a collection of documents. In this case, the collection of documents are sentences in philosophical texts and we are identifying which texts are likely to fall into the category of natural philosophy based on their topics. We will be using Latent Dirichlet Allocation (LDA) to conduct this analysis, which is a statistical model that explains a set of observations (sentences) through unobserved groups (topics), and each group explains why some parts of the data are similar using a set of words. Before the LDA can be run, the dataset first needs to be cleaned.

In [4]:
#Read data
df = pd.read_csv(path_def+'data/philosophy_data.csv')
df.head()
Out[4]:
title author school sentence_spacy sentence_str original_publication_date corpus_edition_date sentence_length sentence_lowered tokenized_txt lemmatized_str
0 Plato - Complete Works Plato plato What's new, Socrates, to make you leave your ... What's new, Socrates, to make you leave your ... -350 1997 125 what's new, socrates, to make you leave your ... ['what', 'new', 'socrates', 'to', 'make', 'you... what be new , Socrates , to make -PRON- lea...
1 Plato - Complete Works Plato plato Surely you are not prosecuting anyone before t... Surely you are not prosecuting anyone before t... -350 1997 69 surely you are not prosecuting anyone before t... ['surely', 'you', 'are', 'not', 'prosecuting',... surely -PRON- be not prosecute anyone before ...
2 Plato - Complete Works Plato plato The Athenians do not call this a prosecution b... The Athenians do not call this a prosecution b... -350 1997 74 the athenians do not call this a prosecution b... ['the', 'athenians', 'do', 'not', 'call', 'thi... the Athenians do not call this a prosecution ...
3 Plato - Complete Works Plato plato What is this you say? What is this you say? -350 1997 21 what is this you say? ['what', 'is', 'this', 'you', 'say'] what be this -PRON- say ?
4 Plato - Complete Works Plato plato Someone must have indicted you, for you are no... Someone must have indicted you, for you are no... -350 1997 101 someone must have indicted you, for you are no... ['someone', 'must', 'have', 'indicted', 'you',... someone must have indict -PRON- , for -PRON- ...

Data Cleaning¶

In [5]:
from nlp_cleaning import cleaner
nlp = en_core_web_md.load()
cleaner(nlp,df,df.sentence_str)
Out[5]:
title author school sentence_spacy sentence_str original_publication_date corpus_edition_date sentence_length sentence_lowered tokenized_txt lemmatized_str cleaned
0 Plato - Complete Works Plato plato What's new, Socrates, to make you leave your ... What's new, Socrates, to make you leave your ... -350 1997 125 what's new, socrates, to make you leave your ... ['what', 'new', 'socrates', 'to', 'make', 'you... what be new , Socrates , to make -PRON- lea... [new, socrates, leave, usual, haunt, lyceum, s...
1 Plato - Complete Works Plato plato Surely you are not prosecuting anyone before t... Surely you are not prosecuting anyone before t... -350 1997 69 surely you are not prosecuting anyone before t... ['surely', 'you', 'are', 'not', 'prosecuting',... surely -PRON- be not prosecute anyone before ... [prosecute, king, archon]
2 Plato - Complete Works Plato plato The Athenians do not call this a prosecution b... The Athenians do not call this a prosecution b... -350 1997 74 the athenians do not call this a prosecution b... ['the', 'athenians', 'do', 'not', 'call', 'thi... the Athenians do not call this a prosecution ... [athenians, prosecution, indictment, euthyphro]
3 Plato - Complete Works Plato plato What is this you say? What is this you say? -350 1997 21 what is this you say? ['what', 'is', 'this', 'you', 'say'] what be this -PRON- say ? []
4 Plato - Complete Works Plato plato Someone must have indicted you, for you are no... Someone must have indicted you, for you are no... -350 1997 101 someone must have indicted you, for you are no... ['someone', 'must', 'have', 'indicted', 'you',... someone must have indict -PRON- , for -PRON- ... [indict, go, tell, indict]
... ... ... ... ... ... ... ... ... ... ... ... ...
360803 Women, Race, And Class Davis feminism But the socialization of housework including m... But the socialization of housework including m... 1981 1981 142 but the socialization of housework including m... ['but', 'the', 'socialization', 'of', 'housewo... but the socialization of housework include me... [socialization, housework, include, meal, prep...
360804 Women, Race, And Class Davis feminism The only significant steps toward endingdomest... The only significant steps toward endingdomest... 1981 1981 117 the only significant steps toward endingdomest... ['the', 'only', 'significant', 'steps', 'towar... the only significant step toward endingdomest... [significant, step, endingdomestic, slavery, f...
360805 Women, Race, And Class Davis feminism Working women, therefore, have a special and v... Working women, therefore, have a special and v... 1981 1981 90 working women, therefore, have a special and v... ['working', 'women', 'therefore', 'have', 'spe... working woman , therefore , have a special an... [working, woman, special, vital, interest, str...
360806 Women, Race, And Class Davis feminism Moreover, under capitalism, campaigns for jobs... Moreover, under capitalism, campaigns for jobs... 1981 1981 199 moreover, under capitalism, campaigns for jobs... ['moreover', 'under', 'capitalism', 'campaigns... moreover , under capitalism , campaign for jo... [capitalism, campaign, job, equal, basis, man,...
360807 Women, Race, And Class Davis feminism This strategy calls into question the validity... This strategy calls into question the validity... 1981 1981 126 this strategy calls into question the validity... ['this', 'strategy', 'calls', 'into', 'questio... this strategy call into question the validity... [strategy, call, question, validity, monopoly,...

360808 rows × 12 columns

We are only using aristotle, so filter

In [6]:
aristotle = df[df.author == 'Aristotle']
In [7]:
aristotle['cleaned_str'] = [' '.join(map(str, l)) for l in aristotle['cleaned']]
long_string = ','.join(list(aristotle['cleaned_str'].values))
wordcloud = WordCloud(background_color="white"
                      , contour_width=0.1
                      , contour_color="black"
                      , max_font_size=100
                      , random_state=42
                      , colormap="Dark2")
wordcloud.generate(long_string)
wordcloud.to_file(path_def+'figs/aristotle_wc.png')
wordcloud.to_image()
Out[7]:
In [8]:
from coherence_modelling import coherence_score_viz
coherence_score_viz(aristotle['cleaned'],'u_mass',1,11,corpora,gensim.models.LdaMulticore,plt,CoherenceModel,path_def)
In [9]:
coherence_score_viz(aristotle['cleaned'],'c_v',1,11,corpora,gensim.models.LdaMulticore,plt,CoherenceModel,path_def)
In [10]:
#https://tedboy.github.io/nlps/generated/generated/gensim.corpora.Dictionary.filter_extremes.html
#https://www.baeldung.com/cs/topic-modeling-coherence-score
In [11]:
#Number of topics
num_topics = 5

#Final LDA model after determining number of topics
id2word = corpora.Dictionary(aristotle['cleaned'])
id2word.filter_extremes(no_below=5, no_above=0.5, keep_n=1000)
corpus = [id2word.doc2bow(doc) for doc in aristotle['cleaned']]

lda_model = gensim.models.LdaMulticore(corpus=corpus,
                                       id2word=id2word,
                                       num_topics=num_topics,
                                       random_state=999)

#Print the Keyword in the 10 topics
pprint(lda_model.print_topics())

[(0,
  '0.015*"come" + 0.015*"thing" + 0.013*"tragedy" + 0.013*"say" + '
  '0.011*"great" + 0.011*"time" + 0.010*"point" + 0.010*"form" + 0.009*"state" '
  '+ 0.008*"speech"'),
 (1,
  '0.018*"animal" + 0.013*"small" + 0.013*"large" + 0.012*"kind" + '
  '0.011*"water" + 0.011*"contrary" + 0.010*"form" + 0.010*"good" + '
  '0.010*"sense" + 0.010*"reason"'),
 (2,
  '0.018*"poet" + 0.018*"man" + 0.017*"animal" + 0.014*"belong" + 0.014*"act" '
  '+ 0.014*"body" + 0.011*"change" + 0.011*"state" + 0.010*"female" + '
  '0.009*"male"'),
 (3,
  '0.021*"thing" + 0.016*"think" + 0.013*"part" + 0.013*"way" + 0.011*"cause" '
  '+ 0.011*"matter" + 0.011*"place" + 0.010*"take" + 0.010*"number" + '
  '0.009*"time"'),
 (4,
  '0.046*"man" + 0.035*"thing" + 0.033*"good" + 0.013*"case" + '
  '0.010*"excellence" + 0.010*"mean" + 0.009*"say" + 0.009*"bad" + 0.009*"law" '
  '+ 0.008*"kind"')]
In [12]:
#Appending the highest probability topic to each sentence
all_topics = lda_model.get_document_topics(corpus, minimum_probability=0.0)
all_topics_csr = gensim.matutils.corpus2csc(all_topics)
all_topics_numpy = all_topics_csr.T.toarray()
all_topics_df = pd.DataFrame(all_topics_numpy)
topic_id_list = all_topics_df.idxmax(axis=1)
aristotle['topics'] = topic_id_list
In [13]:
#Cleaning to create wordcloud
topic_wordcloud = aristotle.groupby('topics')['cleaned_str'].apply(list)
topic_wordcloud[:] = [' '.join(map(str, l)) for l in topic_wordcloud[:]]
topic_wordcloud_df=pd.DataFrame(topic_wordcloud)
topic_wordcloud_df['cleaned_str'] = topic_wordcloud_df['cleaned_str'].str.replace("thing,", "")
In [14]:
#Displaying wordcloud of all the topics
sns.set()
plt.rcParams['figure.figsize'] = [90, 90]

x, y = np.ogrid[:300, :300]

wordcloud = WordCloud(background_color="white", contour_width=0.1, 
                      contour_color="black",  max_font_size=100, random_state=42,
                      colormap="Dark2")

for i in range(5):
    wordcloud.generate(text=topic_wordcloud_df['cleaned_str'][i])   
    plt.subplot(5, 2, i+1)
    plt.imshow(wordcloud, interpolation="bilinear")
    plt.axis("off")
    plt.title(topic_wordcloud_df.index[i], fontdict={'fontsize': 60})
plt.savefig(path_def+'figs/topic_wc.png')
plt.show()

/opt/anaconda3/lib/python3.9/site-packages/seaborn/rcmod.py:400: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if LooseVersion(mpl.__version__) >= "3.0":
/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  other = LooseVersion(other)
In [15]:
#Visualizing intertopic distance and relevant terms of topic
pyLDAvis.enable_notebook()
p = gensimvis.prepare(lda_model, corpus, id2word)
pyLDAvis.save_html(p, path_def+'figs/pyLDAvis.html')
p

/opt/anaconda3/lib/python3.9/site-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import reload
/opt/anaconda3/lib/python3.9/site-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import reload
/opt/anaconda3/lib/python3.9/site-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import reload
/opt/anaconda3/lib/python3.9/site-packages/past/builtins/misc.py:45: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import reload
Out[15]:
In [16]:
from subjectivity_modelling import subjectivity
subjectivity(aristotle, 12, TextBlob)
Out[16]:
title author school sentence_spacy sentence_str original_publication_date corpus_edition_date sentence_length sentence_lowered tokenized_txt lemmatized_str cleaned cleaned_str topics subjectivity
38366 Aristotle - Complete Works Aristotle aristotle When things have only a name in common and th... When things have only a name in common and th... -320 1991 139 when things have only a name in common and th... ['when', 'things', 'have', 'only', 'name', 'in... when thing have only a name in common and t... [thing, common, definition, correspond, differ... thing common definition correspond different c... 4.0 0.550000
38367 Aristotle - Complete Works Aristotle aristotle Thus, for example, both a man and a picture ar... Thus, for example, both a man and a picture ar... -320 1991 56 thus, for example, both a man and a picture ar... ['thus', 'for', 'example', 'both', 'man', 'and... thus , for example , both a man and a picture... [example, man, picture, animal] example man picture animal 4.0 0.000000
38368 Aristotle - Complete Works Aristotle aristotle These have only a name in common and the defin... These have only a name in common and the defin... -320 1991 207 these have only a name in common and the defin... ['these', 'have', 'only', 'name', 'in', 'commo... these have only a name in common and the defi... [common, definition, correspond, different, an... common definition correspond different animal ... 4.0 0.466667
38369 Aristotle - Complete Works Aristotle aristotle When things have the name in common and the de... When things have the name in common and the de... -320 1991 134 when things have the name in common and the de... ['when', 'things', 'have', 'the', 'name', 'in'... when thing have the name in common and the de... [thing, common, definition, correspond, call, ... thing common definition correspond call synony... 0.0 0.500000
38370 Aristotle - Complete Works Aristotle aristotle Thus, for example, both a man and an ox are an... Thus, for example, both a man and an ox are an... -320 1991 52 thus, for example, both a man and an ox are an... ['thus', 'for', 'example', 'both', 'man', 'and... thus , for example , both a man and an ox be ... [example, man, ox, animal] example man ox animal 4.0 0.000000
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
87140 Aristotle - Complete Works Aristotle aristotle ; which is a great advantage, since the more c... ; which is a great advantage, since the more c... -320 1991 137 ; which is a great advantage, since the more c... ['which', 'is', 'great', 'advantage', 'since',... ; which be a great advantage , since the more... [great, advantage, concentrated, effect, pleas... great advantage concentrated effect pleasurabl... NaN 0.589286
87141 Aristotle - Complete Works Aristotle aristotle consider the Oedipus of Sophocles, for instanc... consider the Oedipus of Sophocles, for instanc... -320 1991 118 consider the oedipus of sophocles, for instanc... ['consider', 'the', 'oedipus', 'of', 'sophocle... consider the Oedipus of Sophocles , for insta... [consider, oedipus, sophocles, instance, effec... consider oedipus sophocles instance effect exp... NaN 0.000000
87142 Aristotle - Complete Works Aristotle aristotle There is less unity in the imitation of the ep... There is less unity in the imitation of the ep... -320 1991 308 there is less unity in the imitation of the ep... ['there', 'is', 'less', 'unity', 'in', 'the', ... there be less unity in the imitation of the e... [unity, imitation, epic, poet, prove, fact, wo... unity imitation epic poet prove fact work thei... NaN 0.342857
87143 Aristotle - Complete Works Aristotle aristotle In saying that there is less unity in an epic,... In saying that there is less unity in an epic,... -320 1991 333 in saying that there is less unity in an epic,... ['in', 'saying', 'that', 'there', 'is', 'less'... in say that there be less unity in an epic , ... [say, unity, epic, mean, epic, plurality, acti... say unity epic mean epic plurality action way ... NaN 0.473438
87144 Aristotle - Complete Works Aristotle aristotle If, then, tragedy is superior in these respect... If, then, tragedy is superior in these respect... -320 1991 324 if, then, tragedy is superior in these respect... ['if', 'then', 'tragedy', 'is', 'superior', 'i... if , then , tragedy be superior in these resp... [tragedy, superior, respect, poetic, effect, f... tragedy superior respect poetic effect form po... NaN 0.649345

48779 rows × 15 columns

In [17]:
aristotle.groupby(['topics'])['subjectivity'].mean()
Out[17]:
topics
0.0    0.315566
1.0    0.325995
2.0    0.327974
3.0    0.315465
4.0    0.335072
Name: subjectivity, dtype: float64
In [18]:
df['cleaned_str'] = [' '.join(map(str, l)) for l in df['cleaned']]
subjectivity(df, 12, TextBlob)
Out[18]:
title author school sentence_spacy sentence_str original_publication_date corpus_edition_date sentence_length sentence_lowered tokenized_txt lemmatized_str cleaned cleaned_str subjectivity
0 Plato - Complete Works Plato plato What's new, Socrates, to make you leave your ... What's new, Socrates, to make you leave your ... -350 1997 125 what's new, socrates, to make you leave your ... ['what', 'new', 'socrates', 'to', 'make', 'you... what be new , Socrates , to make -PRON- lea... [new, socrates, leave, usual, haunt, lyceum, s... new socrates leave usual haunt lyceum spend ti... 0.352273
1 Plato - Complete Works Plato plato Surely you are not prosecuting anyone before t... Surely you are not prosecuting anyone before t... -350 1997 69 surely you are not prosecuting anyone before t... ['surely', 'you', 'are', 'not', 'prosecuting',... surely -PRON- be not prosecute anyone before ... [prosecute, king, archon] prosecute king archon 0.000000
2 Plato - Complete Works Plato plato The Athenians do not call this a prosecution b... The Athenians do not call this a prosecution b... -350 1997 74 the athenians do not call this a prosecution b... ['the', 'athenians', 'do', 'not', 'call', 'thi... the Athenians do not call this a prosecution ... [athenians, prosecution, indictment, euthyphro] athenians prosecution indictment euthyphro 0.000000
3 Plato - Complete Works Plato plato What is this you say? What is this you say? -350 1997 21 what is this you say? ['what', 'is', 'this', 'you', 'say'] what be this -PRON- say ? [] 0.000000
4 Plato - Complete Works Plato plato Someone must have indicted you, for you are no... Someone must have indicted you, for you are no... -350 1997 101 someone must have indicted you, for you are no... ['someone', 'must', 'have', 'indicted', 'you',... someone must have indict -PRON- , for -PRON- ... [indict, go, tell, indict] indict go tell indict 0.000000
... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
360803 Women, Race, And Class Davis feminism But the socialization of housework including m... But the socialization of housework including m... 1981 1981 142 but the socialization of housework including m... ['but', 'the', 'socialization', 'of', 'housewo... but the socialization of housework include me... [socialization, housework, include, meal, prep... socialization housework include meal preparati... 0.000000
360804 Women, Race, And Class Davis feminism The only significant steps toward endingdomest... The only significant steps toward endingdomest... 1981 1981 117 the only significant steps toward endingdomest... ['the', 'only', 'significant', 'steps', 'towar... the only significant step toward endingdomest... [significant, step, endingdomestic, slavery, f... significant step endingdomestic slavery fact t... 0.875000
360805 Women, Race, And Class Davis feminism Working women, therefore, have a special and v... Working women, therefore, have a special and v... 1981 1981 90 working women, therefore, have a special and v... ['working', 'women', 'therefore', 'have', 'spe... working woman , therefore , have a special an... [working, woman, special, vital, interest, str... working woman special vital interest struggle ... 0.485714
360806 Women, Race, And Class Davis feminism Moreover, under capitalism, campaigns for jobs... Moreover, under capitalism, campaigns for jobs... 1981 1981 199 moreover, under capitalism, campaigns for jobs... ['moreover', 'under', 'capitalism', 'campaigns... moreover , under capitalism , campaign for jo... [capitalism, campaign, job, equal, basis, man,... capitalism campaign job equal basis man combin... 0.438889
360807 Women, Race, And Class Davis feminism This strategy calls into question the validity... This strategy calls into question the validity... 1981 1981 126 this strategy calls into question the validity... ['this', 'strategy', 'calls', 'into', 'questio... this strategy call into question the validity... [strategy, call, question, validity, monopoly,... strategy call question validity monopoly capit... 0.000000

360808 rows × 14 columns

In [19]:
df.groupby('school').agg({'subjectivity': ['mean']})
Out[19]:
subjectivity
mean
school
analytic 0.286684
aristotle 0.341157
capitalism 0.367580
communism 0.290665
continental 0.314276
empiricism 0.357706
feminism 0.361778
german_idealism 0.312720
nietzsche 0.333113
phenomenology 0.283388
plato 0.319806
rationalism 0.366719
stoicism 0.329450
In [20]:
abstracts = pd.read_csv(path_def+'data/covid_abstracts.csv')
abstracts.head()
Out[20]:
title abstract url
0 Real-World Experience with COVID-19 Including... This article summarizes the experiences of COV... https://pubmed.ncbi.nlm.nih.gov/35008137
1 Successful outcome of pre-engraftment COVID-19... Coronavirus disease 2019 COVID-19 caused by... https://pubmed.ncbi.nlm.nih.gov/35008104
2 The impact of COVID-19 on oncology professiona... BACKGROUND COVID-19 has had a significant imp... https://pubmed.ncbi.nlm.nih.gov/35007996
3 ICU admission and mortality classifiers for CO... The coronavirus disease 2019 COVID-19 which ... https://pubmed.ncbi.nlm.nih.gov/35007991
4 Clinical evaluation of nasopharyngeal midturb... In the setting of supply chain shortages of na... https://pubmed.ncbi.nlm.nih.gov/35007959
In [21]:
cleaner(nlp,abstracts,abstracts.abstract)
abstracts['cleaned_str'] = [' '.join(map(str, l)) for l in abstracts['cleaned']]
In [22]:
subjectivity(abstracts, 4, TextBlob)
abstracts['subjectivity'].mean()
Out[22]:
0.4210568621231637

A sentence like 'The impact of the coronavirus disease 2019 (COVID-19) pandemic on well-being has the potential for serious negative consequences on work, home life, and patient care.' found in one of the abstracts, is actually a subjective sentence.

In [23]:
https://levelup.gitconnected.com/simple-nlp-in-python-2cb3243239d3

  Input In [23]
    https://levelup.gitconnected.com/simple-nlp-in-python-2cb3243239d3
          ^
SyntaxError: invalid syntax